WEAKLY CONVEX SETS AND MODULUS OF NONCONVEXITY 

MAXIM V. BALASHOV AND DUSAN REPOVS 

Abstract. We consider a definition of a weakly convex set which is a generalization of the notion of a 
weakly convex set in the sense of Vial and a proximally smooth set in the sense of Clarke, from the case of 
^"^ , the Hilbert space to a class of Banach spaces with the modulus of convexity of the second order. Using the 

^«. 1 1 new definition of the weakly convex set with the given modulus of nonconvexity we prove a new retraction 

theorem and we obtain new results about continuity of the intersection of two continuous set- valued mappings 
(one of which has nonconvex images) and new affirmative solutions of the splitting problem for selections. 
We also investigate relationship between the new definition and the definition of a proximally smooth set 
and a smooth set. 
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1. Introduction 



We begin by some definitions for a Banach space {E, \\ ■ ||) over M. Let Br{a) — {x £ E \ \\x — a\\ < r}. 

2 ' Let cl A denote the closure and int A the interior of the subset A <Z E. The diameter of the subset A <Z E 

is defined as diam^ = sup ||a; — y\\. The distance from the point x ^ E to the subset A <Z E is defined as 

g(x, A) = inf lla; — all. For a subset A d E, let Ud(A) be the open d-neighborhood of A, i.e. 



Ud{A) = {x^E I q{x,A) <d}. 
The Hausdorff distance between two subsets A, B d E is defined as follows 

h{A, B) = max < sup g{a,B), sup g{b,A) 

laeA beB 



o, 

f^ I We denote the convex hull of the set A by coA. 

Definition 1.1. (0, [TU]) Let xo,xi G E, ||a:i-a;o|| < 2d. The set 

,^, Dd{xo,xi)= Pi Bd{a) 

rS ' aeE: {xo.xi}CBd(a) 

5t 1 is called a strongly convex segment of radius d, and the set 

D°^[xo,xi) = Dd{xii, xi) \ {xo, xi). 

is called a strongly convex segment of radius d without extreme points. 

Definition 1.2. (Vial [H], see Figure 1). A subset ^4 of a normed space is called weakly convex (in the 
sense of Vial) with constant i? > 0, if for any pair of points X{),xi G A such that < ||a;i — a:;o|| < 2R the 
set A[\D°j^{xq,xi) is nonempty. 

Definition 1.3. (Clarke et al [7], [8]). A subset A oi a normed space E is called proximally smooth with 
constant d > 0, if the distance function x — >■ q{x, A) is Frechet differentiable on the tube Ud{A)\A. 

Definition 1.4. (Polyak [17]) Let E he a, Banach space and let a subset A d E he convex and closed. 
The modulus of convexity 5a ' [0, diamA) -^ [0, +c») is the function defined by 

' C A, Vxi, X2 £ A : \\xi — X2\\ — e 



(SA(e) = sup<^(5>0 
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2 MAXIM V. BALASHOV AND DUSAN REPOVS 

It is obvious that SaIO) — 0. 

Definition 1.5. (Polyak |j/7j, see Figure 2). Let _E be a Banach space and let a subset A C E he convex 
and closed. If the modulus of convexity Sa{£) is strictly positive for all e G (0, diamA), then we call the set 
A uniformly convex (with modulus Sa{-))- 

We proved in [3, that every uniformly convex set A ^ E is bounded and if the Banach space E contains 
a nonsingleton uniformly convex set A j^ E then it admits a uniformly convex equivalent norm. We also 
proved that the function e — >■ SA{s)/e is increasing (see also [14[ Lemma I.e. 8]), and for any uniformly convex 
set A y^ E there exists a constant C > such that 5a{s) < Ce^ [3]. 

Let 6e be the modulus of convexity for the Banach space E, i.e. the modulus of convexity for the closed 
unit ball in E. 

Definition 1.6. Let £' be a Banach space. Let a subset A C E he closed and d G (0,diamA). The 
modulus of nonconvexity ja ■ [0, d) — > [0, +oo) is defined as 



7A(e) = inf <^ 7 > 



_D-v 



Xl + X2 



nA^?),yxi,X2eA: ||xi -a;2|| < £ 



and7A(0) =0. 

It is easy to see that the modulus of nonconvexity is a nondecreasing function. Besides, we shall further 
suppose that the modulus of nonconvexity is continuous from the right. Otherwise we shall redefine the 
modulus by continuity from the right. 

Definition 1.7. (see Figure 3). Let £' be a Banach space, and let a subset A C E he closed. We shall 
call the set A weakly convex with modulus of nonconvexity 7^1 (e), e £ [0, d) {d < diam A), if the modulus of 
nonconvexity ja satisfies the inequality 

0<7a(£)<|, Ve€[0,d). 



D'-U^'^i;x2}, 
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Figure 1 




Figure 2 



Figure 3 



It is obvious that the equality 7a (e) — for all e G [0,diam^) means (for the closed set A) convexity of 
the set A. 

Hereafter the text "weakly convex" means weakly convex in the sense of Definition 1(71 

Example 1.1. Let E = H he the Hilbert space and (5-^(6) = 1 — w 1 — ^ be the modulus of convexity 
of H. A weakly convex subset A C H with modulus 7a (e) — dS-u{s/d), e £ [0, d), d > 0, is weakly convex in 
the sense of Vial with constant d and proximally smooth with constant d (see [51 [71 HI [IHl [13 : in particular 
[2]). These three properties are equivalent in the Hilbert space. 

The relationship between weak convexity in the sense of Vial and proximal smoothness of a set in a 
Banach space is much more complicated (see [2 for details). 

The next lemma is a simple consequence of similarity. 

Lemma 1.1. (\T6\ Lemma 2.7.1JJ Let a space E be uniformly convex with modulus Se- Then for all 
x,y E Bi{0), such that \\x — y|| = e > 0, and for any /3 G (0, 2] the following inequality holds 



B2ps,(e){{l~P)x + Py)cB,{0). 
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Lemma 1.2. Let a space E be uniformly convex with modulus 6e- Then for any s,r] such that < e/2 < 
77 < e < 2 the following inequality holds 



< 



V 



SEirie)), 



77 e e • 77 

where r(e) = j (^ - fe(e)). 

By the Day-Nordlander theorem [9], ^^(e) < ^ < e/2 for aU e G (0, 2). Hence r(e) > for ah e e (0, 2). 

Proof. Let's fix e e (0,2), a € (0, | — ^Se{£)) and A £ (0, 1). Choose points a;i,a;2 G B-^(O), such that 
ll^^i — X2\\ — e and 5e{£) + a> 5, where 5 = sup{r > | Br{z) C -Bi(O)} and z — ^{xi+ X2)- 

For any natural number k we define the point Ok G B]^(0) with the property ||afe — z\\ < (5 + ^7. Let y^ be 
the homothetic image of the point Xi under the homothety with center Ok and coefficient A, i = 1, 2; let Zk 
be the homothetic image of the point z under the homothety with center Ok and coefficient A. 

By construction, || 2/1 — 2/2 li = ^^ ^"^^ \\zk — cik\\ < A(5 + A^ . By the triangle inequality and by the property 
of chosen points Xi, Ok we have \\xi — ak\\ > \\ ^^~^^ \\ — \\ak — z|j > -j (| — Sa{£)) — r{e) > for i = 1,2 and 
sufficiently large k. Let /3 — min{A, 1 — A} e (0, 5]. By Lemma llT]we have B2psE{rie))iy2) <^ -Bi(O), i = 1, 2. 
Hence 



SsiXe) < 



Vi 



^2 



flfe 



- 2pSE{r{e)) = \\zk - ak\\ - 2/3(5£(r(e)) < XSeie) + Xa + X- - 2l3SE{r{e)). 



Letting a — ?► +0, fc — )■ 00, we obtain 



dE{Xe)<X6E{e)-2pdEir{s)). 



The desired estimate appears if we put A = rj/e. D 

One of the important motivations for consideration of weakly convex sets in the sense of Definition 117] is 

given by the next theorem. 

Theorem 1.1. Let a space E he uniformly convex with modulus 5e, d > 0. Let A d E be a weakly 

convex set with modulus of nonconvexity ja, and suppose that function ddEis/d) — 7a (e) is positive for all 

£ G (0, min{2(i, diamyl}). Then for any point x G Ud{A) the set 

Pax = {a G a I ||x - a|| = g{x, A)} 

is a singleton. 

Proof. {l).Nonemptiness of Pax. Let's fix x G Ud{A)\A. Let points a^ G A be such that ||a; — ak\\ — > 
g{x^ A). Define nonnegative numbers £k = \\x ~ ak\\ — q{x, A). 

Suppose that the sequence {ofcl^i has no converging subsequence. Without loss of generality we may 
assume that there exists a number eg > such that for any natural k^m the following inequality holds: 
||afe — flmll > Eo- By the definition of £fc, e™ we have 



max{||a;-afc||,||a;-am||} <£k+£m + q{x,A). 



Let Q = g{x, A). Then 



< e + £k 



{g + £k+ £m)6E{\\ak - am\\/{g + £k + £m))- 



Due to the weak convexity of the set A for any a > there exists 

' Ok + a,r. 



Hence ||a; — Ofe^ 



< \\x- 



ak+a-rr 



Okm e B^^(^\\ak-a,„\\)+a 

+ 7A(||afe -a„i\\) + a< 



r\A. 



< g + £k + £ni + a - {g + £k + £m)5E{\\ak - am\\ / {q + £k + £m)) +7A(||afc-am||)- 

Let's choose di G {^d,d) and a sequence ak > 0, a^ — >■ 0, such that for all sufficiently large fc,m the 
inequality g + ek + £m + ak < di < d holds. Then by Lemma 1(2] 



{g + £k +£m)SE{\\ak ~ am\\/ig + £k +£m)) > dlSE 



di 
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and we have the estimate ||a; — akm\\ < Q + £k + £m + cik— 

By Lemma ll2]it fohows that 

d,SE (l^h-^^ d5E (l^h-^^ > 2{d - d^)SE{r{\\a, a„,\\/d,)). 

From the mequahties £o < \\<^k — o-mW < 2(ii and r{e) = I (| ~ fel^)) ^ i ( § ~ T ) > 0: it fohows that 
for ah fc,m the value SE{r{\\ak — am\\/di)) is bounded from below by a positive constant c > 0. Hence for 
sufhciently large k^m (when £k + £m + ctk < 2(d — di)c), \\x — akmW < Q- Contradiction. Therefore, the 
sequence {afcjfc^i has a converging subsequence and Pax ^ 0. 

(2). The set Pax is a singleton. The proof is similar to the step 1. If g(x, A) = ||a; — ai||, i = 1,2, ai, 02 G A, 
then we have 

Oi + 02 



and for all a > 



< g{x, A) - q{x, A)5E{\\ai - a2\\/ q{x, A)), 



da e -D7^(||ai-a2||)+a I ' I 11 ^• 



Now by choosing < a < q{x,A)5e{\\o.i — a2\\/Q{x,A)) — 7A(||ai — a2||), we obtain that ||a; — a\\ < 
\\x - ^^^\\ + -fA{\\ai - a2\\) + a < 

< gix,A) - g{x,A)SE{\\ai - a2\\/g{x,A)) +jA{\\ai - a2|l) +a < g{x,A). 

Contradiction. D 

By Theorem l[T]and the results from [2^ it follows that if the space E is additionally uniformly smooth 
then each weakly convex set with the modulus 7^ (for which ddEi^/d) — 7a (e) > 0) is proximally smooth 
with constant d > 0. We note that d is not the largest possible constant for the proximal smoothness of the 
set A. 

It's easy to see that the proximal smoothness with constant d > implies the weak convexity. Suppose that 
(for simplicity) the subset A d E is compact in the strong topology of the Banach space E and proximally 
smooth with constant d > 0. Then the set A is weakly convex with some modulus of nonconvexity 7A(e), 
e e (0,min{2(i, diam^}). Indeed, the compactness of the set A implies that the values of modulus from the 
Definition lIHlare achieved for every e G (0, min{2d, diam^}) at some points ai, 02 S A, \\ai — 02!! < £■ This 
means that for the point x = i(ai + a2) we have g{x,A) = 7_A(e) > 0. Using inequality g{x,A) < ^e, we 
obtain from the estimate ^e < d and from proximal smoothness of the set A (see [H Theorem 2.4]) that the 
set Pax is a singleton and g{x,A) ~ Ja{£) < ^£- 

2. The order of function ja 

Before further considerations we shall make some remarks. Consider for simplicity a set A on the Euclidean 
plane. Let the boundary A be a smooth closed curve x — x{s), y — y{s), where s is the natural parameter. 
Suppose that the curve A contains no straight segments. In this case the radius of curvature of A at the 
point (x(s),y(s)) equals R{s) = (a;'^(s) +y'^(s))^/^/|x"(s)y'(s) - y" {s)x' {s)\. If the radius R{s) is finite and 
positive at the point {x{s),y{s)) (and this takes place for a.e. values of parameter s), then the curve at the 
neighborhood of the point {x{s), y{s)) is similar to the circle of radius R{s). 

If additionally, the set A is not locally convex at the point {x{s),y{s)) (i.e. for any r > the set 
An Br{{x{s),y{s))) is nonconvex), then for a small e > the function 7a (e) has the order no smaller than 

£^ (more precisely, 7a(£) > R'^i^) ~ y R'^is) ^ i")- 

We shall show that the situation above is typical: if the set A is nonconvex, then the modulus of non- 
convexity for A satisfies the estimate 7a(£) > Const • e^. As we have mentioned above the modulus of 
nonconvexity for convex set A equals zero. 
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If the subset A of the Banach space E is a. symmetric cavern (i.e. A = cl{E\B), where B is a closed 
convex bounded symmetric body), then 7^1 (e) > Const • e^. The proof follows by the fact that in this case 
the function 7a (e) has the same order as the the function 

<Je,b{£) ^sup<l \x,yeB, \\x-y\\B<e 

introduced in 4l. Here || • \\b is the norm in the space E with the unit ball B. In |4] and [5] the inequality 



1— W 1 — ^ < aE,B{£) was proved. In fact, it is the "dual" of the Day-Nordlander theorem. It can be proved 
similarly as the Day-Nordlander theorem (see [SJ §3, pp. 60-62] for details). The proof is the same except that 
instead of function 5x(£) — hif^ A(e, ip) one should consider on page 62 the function axis) = sup^ A(e:, ip). 

Let the subset A C E from a Banach space _E be a cavern, i.e. A = c\ {E\B) where the set i? C -B is a 
closed convex and bounded body, £ int B. We shall estimate the value of 7a (e)- 

For any closed convex and bounded set B d E, £ int i?, we define the Minkowski function 

^ib{x) =mi{t>0\xetB}, ^x <E E. 
For any bounded set C C -E we define B- diameter of the set C as follows 

diamsC= sup ^B{x — y). 

x,yGC 

For any closed convex bounded sets A, B , C C E we define the modulus 

a^^^ie) = inf (a > I (aA + ^^-^] D {E\mt C) 7^ 0, yx,y £ A : figix - y) < e 



and the modulus 



ac{e)^4'^''^'''''\e). (2.1) 



/ID 

Moduli CTp' and ac generalize the definition from 4 to arbitrary convex sets. 

It is obvious from the definition of ac that if C is a convex body then we have for all admissible e > 

for the set A = cl iE\C) = cl {E\mt C) 

7A(e) = ac{e). (2.2) 

The next lemmas are direct consequences of the definition of a^-,' . 

Lemma 2.1. For any bounded closed convex bodies A, B,C G E and t > 0, the following holds: 

(1) dp' {e) — -a^j' (e), Ve G (0,diamBC); 



(2) i7(^'* [e) — a(j' (ts), Ve e [ 0, - diaia bC ] ; and 



1 
J 

(3) f^w^i^) = tc^c^ (-) , Ve e (0,tdiamBC) . 

Lemma 2.2. For any bounded closed convex bodies A' , B' , A, B , C G E and e € (0, diamsC), 

(1) if A' C A then a^ ' (e) > a^j' (e); and 

(2) ifB'cB then a^'^' {e) < CTc-^(e). 

Theorem 2.1. Suppose that the subset A d E of a Banach space E is a cavern. Let Br{0) C cl {E\A) C 
-Bfl(O). Then for all e G (0, 2r) we have 

lA{e) > ^r. 



Proof. Let B = c\ {E\A) be a closed convex body. Using Lemmas 2lT]and 2l2]we get 

aB{e) = .T'^^^''\e) = r.f^^^^^^^^^) (1) > r^r (£ 

Using the result of Banas W we obtain 

.2 



"^■''(^)^"^(^) = i-r-:o?^-^- 
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By invoking formula {2^ we complete the proof. D 

Of course, a weakly convex set is not necessarily a cavern. But if such set A is connected and nonconvex, 
then it has "cavern-like" parts, and hence 7a (s) > Const • e^. 

Hereafter all Banach spaces will have the modulus of convexity of the second order at zero and will contain 
weakly convex sets with modulus of nonconvexity of the second order, too. There are many such spaces 
besides Hilbert spaces, for example Ip, p £ (1,2) (see [2], [5], [5] for details). 

We shall define a special condition. 

Definition 2.1. Let 6 be the modulus of convexity for some closed convex set A and 7 the modulus of 
nonconvexity for some closed weakly convex set B. We shall say that condition (i) is valid for the moduli 5 
and 7 if 

(1) for all s e [0, So] there exists a solution t — tg > s oi the equation 5{t — s) — j{t) = 0, 

(2) the function S{t — s) — "f(t) is positive and increasing for t > ts and 

(3) there exists a solution t{s) of the equation 5{t — s) — j{t) = |. 

In the case when 6{t) = d6E{t/d) is the modulus of convexity for the ball Bd{0) in the Banach space E 
we shall define the solution t{s) as i£;(s). 

Remark 2.1. Definition 2lT]has a technical character (it is useful for further proofs) and it is not so 
exotic. Suppose that for sufficiently small t > our moduli have the second order at zero and are defined 
by formulae S{t) = cit^ + o(t^), t -^ +0, ^{t) = C2t^ + o(i^), t -^ +0, and ci > C2 > 0. Then for sufficiently 
small numbers s > the function t -^ 6(t — s) — 7(i) is positive and increasing, and t{s) x yG, s — >■ +0. 

Remark 2.2. Suppose that a Banach space E has modulus of convexity Se of the second order and 
a closed subset A d E is weakly convex with the modulus of nonconvexity 7^ of the second order, too. 
Taking into account that for all < d < di ^^^(e/d) > di5E{£/di), Ve G {0,2d) (see [3J Lemma 2.1]), and 
dSsis/d) X ^, e — >• +0, we can conclude that there exists a number d > such that dSEi^/d) > ja{£), 
Ve G (0, 2d). If additionally the space E is smooth then by Theorem lUand [2j Theorem 2.4] we obtain that 
the set A is proximally smooth with constant d. 

3. Properties of weakly convex sets 

Theorem 3.1. Let d > 0. Let a subset A of a Banach space E be weakly convex with modulus 7yi(£), 
£ G [0,d) and the subset B <Z E be uniformly convex with modulus 5b (e), e G [0, diamS) and diami? < d. 
Let 5b{£) > 7a (e) for all e G [0, dianiB). Then the set An B, if nonempty, is weakly convex with modulus 
7AnB(e) £ 7-A(e), £ € [0,diamAn B) and connected. 

Proof. The weak convexity of the intersection and the estimate for the modulus follows by definitions. 

Suppose that the set AO B is not connected. This means that there exist two nonempty closed disjoint 
sets Ai C An B and A2 ^ {An B)\Ai. Choose k = 1 and points ai € Ai, bi € A2. 

Due to weak convexity of the set An B there exists a point w E ^(a^ + bk) + (7A(||afc — bk\\) + ctk)Bi{0), 
w £ A. The numbers at are chosen by the conditions afc — )■ and < 0;^ < ^(|||afc — bk\\ — 7A(||afc ^ ^fcll))- 

One of the inclusions w £ Ai or w £ A2 is true, li w £ Ai, then denote Ok+i = w, bj^+i = bk- li w £ A2, 
then denote Ok+i = a^, bk+i = w. In this way we build the sequences {afcj^i C Ai, {&fc}^i C A2. 

Let Ik = \\ak — bk\\. Then < Ik+i < ^h + 7A(^fc) + ctk < h- Hence Ik ^ I > 0- Taking the limit fc — > 00 
and using continuity of the function 7a from the right we get ^l < 7a(0- I^ follows from Definitions lIHland 
iHthat I = 0. Therefore, \\ak - bk\\ -^ 0. 

We proved in [^ that for any uniformly convex set B there exists a number c > such that the modulus 
of convexity for the set B can be estimated as follows 5b{£) < ce^, e £ (0,diam_B). 

It follows from the construction of points Ok+i that 0^+1 ~ Ok, or 

1 3 c 

||afc+i - flfcll < -\\ak -bk\\+ lA{\\ak - bk\\) + ak < -rh + -^ll- 

In the latter case 6i._i 1 — bi. and 



fe+i — "fc 



h+i = hk+i - bkW < -,lk + ^^fc = ( 7 + ^^fc ) h < ^Ik, 



3, c,2 /3 c, \ , ^ 4, 

42*^ ^4 2/5 

for all k > ko. Thus there exists a number d > 0, such that ^fc < d (|) .It follows from the estimate 

3d /4\'' c JiV" /4^ '' 



\ak,.-ak\\<-i- +-d - <K 
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which is vahd for sufBciently large fc, that for such k and m > k 



m— 1 m — 1 //|\" / A^ ^ 



\am - Ofcll = ^ ||a„+i - a„|| <^K [j] < 5X ( - 

n=fc n— fc 



the latter means that the sequence {at} is fundamental. Since ||a/j — 6fc|| — t- thus the sequence {bk} is also 

fundamental. By the closedness of the sets Ai and A2 and from the condition ||afc — &fc|| — >■ we conclude 

that Gk ^ X e Ai, bk ^ X e A2. Hence Ai n ^2 ^ 0- □ 

For any closed subset A C E, a point x G Ud{A) and a number s > we define the set-valued projection 

Pa{x,s) ^{ae A\ ||a;-a|| < g{x,A) + s}. 

It follows by definition that Pa{x,s) ^ for all s > 0. Apart from this, under conditions of Theorem lHJ 
Pa{x,0) — Pax is a singleton. 

Theorem 3.2. Let a Banach space E be uniformly convex with modulus 5e- Let a subset A <Z E be 
weakly convex with modulus 7a (e), £ G [0, 2d). Let dSEi^/d) > 7a (e) for all e £ (0,2d). Suppose that the 
condition (i) from Definition 2{^ is satisfied. Then 

Pa{x,s) C Btj,(^s){PAx). 

Proof. Let a = Pax and b G Pa{x,s), s < sq. Let's define the point y G [b,x] by the condition 
Ik -2/11 ^g{x,A)^ \\x-a\\. Let w = ^, z = ^. 

It follows from the inequality \\b — y\\ < s that \\w — z\\ < s/2. In the triangle bya we see that ||y — a\\ > 
\\a-b\\-s. 

Let Q = g{x,A). Note that \\a — 6|| — s < \\y — a\\ < 2g < 2d. If the inequality gSE{{\\a — b\\ — s)/g) — 
7A(||a — 6||) > I holds, then for some a > we have gSE{{\\a — b\\ — s)/q) — 7A(||a — b\\) > | + a. Using the 
inequality gSE{{\\a — b\\ — s)/g) < gSE{{\\a— y\\)/g), we obtain 

Oq G i3-y^(||a-6||)+a(w) n A, i3^_^(||a_6||)+Q,(w) C mt B gSj^j^Q\a-b\\-s)/ g){z)- 

Hence ||ao — x\\ < ||ao — w\\ + \\w — z|| + ||z — x\\ < 

< 7A(||a -b\\) + a + ^ + g{x, A) - g{x, A)SE{{\\a - b\\ - s)/g{x, A)) < g{x, A). 

This contradiction shows that dSE{{\\a — b\\ — s)/d) — -fA{\\a — b\\) < gSE{{\\a — b\\ — s) / g) — jAiWa — b\\) < | 
and by the conditions of the theorem, ||a — 6|| < tE{s). The point b G Pa{x, s) was arbitrary and the theorem 
is thus proved. D 

Corollary 3.1. Under the assumptions of Theorem 50 the projection Pax uniformly continuously 
depends on X. More precisely, «/ ||a;i — a;2|| < Sq and xi,X2 £ Ud{A), then \\ Pax i — PaX2\\ <tE{\\xi — X2\\). 
Moreover, tE{s) x ^/s, s — > +0. 

Theorem 3.3. Suppose that the assumptions of Theorem 3\^hold and di G (0,d). Then for any point 
x £ E the set AnBd^^^x), if nonempty, is weakly convex with modulus jAnBa {x){£) ^ 7a(£)7 e G [0, diamAn 
Bdiix)), and path connected. 

Proof. Weak convexity of the intersection follows from the definitions. 

Fix any pair of points x,y £ A such that < ||a; — y|| < 2di. For any number t G [0; 1] we denote 
zt = (1 — t)x + ty. The map z 1-^ Paz is single-valued and continuous (Corollary 3[T|) on the set Udi{A), 
hence it is single- valued and continuous on the set U' — Ud^ {A) IJ A. Since zt G U' for all t G [0; 1] there is 
a unique point a{t) with {a(i)} = Pazi- The function a : [0; 1] — > A is continuous and defines the desired 
curve r = {a{t) : i G [0; 1]} which connects points x and y. D 

Theorem 3.4. Let E be a uniformly convex space with modulus Se- Let A d E be a weakly convex set 
with modulus of nonconvexity 7A(e), e G [0, diamA). Let A C Br{a), 2r < d and 7a (t) < ddEi^/d) for all 
£ G [0,diam^). Suppose that the assumptions of Theorem 3\Mhold. Then the set A is a continuous retract 
ofE. 

Proof. Let x G E\A. Let B = c\ co A C Br{a), y = Pbx. Due to the uniform convexity of the space E 
the metric projection on the set B is continuous. We observe that this projection is uniformly continuous 
(see [13] and O Example 3.2]) on the balls. 
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y = Pbx 
z = PaV 



Figure 4: Scheme of retraction 

Since y = y{x) G B C Br{a) we have g{y, A) < 2r < d. By Theorem 1(1] and Corollary 3(1] there exists a 
unique metric projection z = PAy which uniformly continuously depends on y. Therefore, z{x) = Pa{Pbx) 
is the desired retraction, see Figure 4. D 

Remark 3.1. We remark that function z{x) from Theorem 3|4]is uniformly continuous on the balls. 

Let us also mention that Theorem 30] remains valid in any uniformly convex and smooth Banach space 
for any proximally smooth set A with constant d and A C i?r(a), rf < 2r. Instead of Theorem 1(1] and 
Corollary SHJone must use the results from [21 Theorem 2.4]. 

Theorem 3.5. Let {T,p) be a metric space. Let Fi,F2 '■ {T,p) — >■ 2^ be set-valued mappings, con- 
tinuous in the Hausdorff metric. Suppose that for a point to Cz T the set i^i(io) is uniformly convex 
with modulus S{e), and the set i^2(io) is weakly convex with modulus 7(e). Let 7(e) < (5(e) for all e < 
min{diamFi(to), diami^al^o)}- Let H{t) = Fi(t) n F2{t) ^ for all t e T. Then the mapping H(t) is 
continuous at the point t = to in the Hausdorff metric. 

Proof. It follows from the uniform convexity of the set Fi(io) that it is bounded ([H], [31 Theorem 2.1]). 
Due to the continuity in the Hausdorff metric we conclude that there exists a number i5 > such that the 
set cl U F{t) is compact. By the Closed Graph Theorem [T] the set-valued mapping H{t) is upper 

p(t,to)<<5 

semicontinuous at the point t ~ to, i.e. 

Ve > 35 > Vt : pit, to) < S H{t) c H{to) + B,{0), 



lim sup H{t) C H{to) 

t-Ho 



(3.3) 



If the set H{to) is a singleton then the continuity of the set-valued mapping H at the point to follows by 
its upper semicontinuity. Next we shall assume that the set H{to) consists of more than one point. 
Suppose that lower semicontinuity fails, i.e. that 

H{to) (t lim inf H{t). 
Thus there exist a number eo > and points tk (z T, tk -^ to, such that 

H{to) (t H{tk) + -Beo(0)j for any natural k. 

For any k there exists a point hk G H{to) with 

hk^H{tk) + B,M- 
Since the set H{to) is compact, thus without loss of generality we may assume that hk — > ho £ H{to) and 



ho ^ H{tk) + Bi,^/2{0), for any natural k. 



(3.4) 



Let us define the set Ho = lim sup H{tk) C H{to). By construction ho £ H{to)\Ho, hence Ho ^ H{to 
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Let xq £ -Pfffj/io- For ho e Fi(to) n ^2(^0) and xq G Hq put I = \\ho — xo\\ > 0. 
Using uniform convexity of i^i(io) we get 

Due to the weak convexity of the set F2{to) and finite dimension of images of the mapping F2 we can find a 
point 



/es,(,(^)nf2(^„) 



By continuity of the map F2 there exist points fk € F2{tk) with /^ — > /. Besides, for e = {S{1) — 7(/))/3 we 
can find a natural number ko, such that for all k > ko the following holds: 

A.Gi?,(,)_, (^^^)cFi(tfe), (3.5) 

and 

/feei?,(o+. (^4^)nF2(M. (3.6) 

By the formulae (SjS]) and (SlH]) it follows that fk G H{tk) for all k > ko. Hence / — lim /^ G -ffo- 
At the same time 



\ho-f\\< 



, -To + /ic 
ho — 



_ Xq + ho 



< 211^0 ^'^oll+ldl^^o- /loll) < \\xo-ho\' 



This contradicts with the inclusion xo G Piioho. Thus, H{t) is lower semicontinuous at the point t = to- □ 
Let F : (T,p) — > 2'^^'IIH) be a set-valued mapping. If for any t € T the set F(t) is uniformly convex 

with modulus Spu\{e) > S{e), e G [0,diam_F(t)), and 6 is an increasing function, then we shall say that the 

set-valued mapping F is uniformly convex with modulus d. 

If for any t £ T the set F{t) is weakly convex with modulus of nonconvexity 7_F(t)(e) < 7(e): £ G 

[0,diamF(i)), 7(0) = 0, 7(e) < | for admissible e > and function 7 is continuous from the right and 

nondecreasing then we shall say that the set-valued mapping F is uniformly weakly convex with modulus 7. 

Definition 3.1. Let a set-valued mapping Fi : {T,p) — > 2*^-^'"H) be uniformly convex with modulus 5 
and a set-valued mapping F2 : (T, p) —> 2^^''''''' uniformly weakly convex with modulus 7. We shall say that 
condition (ii) is valid for the moduli 5 and 7 if 

(1) for all s £ [0, So] there exists a solution t = ts > s oi the equation 5{t — s) — ^f{t) = 0, 

(2) the function 5{t — s) — ^{t) is positive and increasing for t > t^, 

(3) there exists a solution t{s) of the equation 5{t — s) — j{t) = |. 

It follows from the results of the second paragraph that condition (ii) is possible only if moduli S and 7 
are of the second order at zero. 

We say that set-valued mapping F is uniformly continuous with modulus of continuity a; > if for any 
ii,i2 e T the inequality h{F(ti), F{t2)) < uj{p{ti,t2)) holds. 

Theorem 3.6. Let Fi,F2 : {T,p) — > 2^^'"'"'. Let the values of F2 be uniformly convex with modulus (5(e). 
Let the values of Fi be uniformly weakly convex with modulus 7(e). Suppose that set-valued mapping Fi is 
uniformly continuous with modulus UJi, i = 1,2. Let the condition (ii) holds. 

Let H{t) ~ Fi(t)nF2(t) ^ for allt £ T and suppose that for some M > the inclusion [J H(t) C -Bm(O) 

tGT 

holds. Then 

HHih), H{t2)) < ( lU tr ^'.&:l ' ''^ < ^°' (3.7) 

L so ^ 2 — ''O- 

Here w^ = LUi{p(ti,t2)), i = 1,2. 

Proof. Let ci G H{ti). We shall show that for any number A, which is strictly larger than the right side 
of the formula (SlT]), there exists a point C2 G H{t2) with ||ci — C2II < A. This will prove the theorem. 
Fix d £ H{t2). If LOi + UJ2 > 2so, then, taking C2 = d, we obtain that 

Wl -I- UJo 

h{H{ti),H{t2))< \\ci-C2\\ <2M<— -M. 

So 

Suppose that uji + uj2 < Ssq. Fix A: > 1, such that inequality kuji + k^uj2 < 2so holds. For the point 
ci G H{ti) = Fi(ti) n i^2(ii) we can find the point b £ ^2(^2) such that \\b- ci|| < kuj2- 
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Fix the point 6^ G -^1(^2) which satisfies the condition \\b— b,r\\ < k ■ g{b,Fi{t2))- Invoking the inequahty 
p(6, Fi(ti)) < ||5— cill < kuj2 we get the following estimate 

\\b-b^\\ <kg{b,Fi{t2)) < kh{Fi{ti),Fi{t2)) + kg{b, Fi{ti)) < /cwi + /c||6 - ci|| < /cwi + /cV • 

Define the point a e [d,b] n H{t2) as the one which is nearest to the point b. The set [d, 6] n H{t2) is 
nonempty because it contains the point d. Put n = 1, ai = a. 
Consider the following cases: 



(1) (5(||a„-6||) >7(l|an-&7r 

(2) 5(||a„-6||)<7(l|an-&.,i; 1 21 
If the case (1) takes place then we choose a„ 



h\\b-b^ 
Mb-b^ 



min { -, - iS{\\an - b\\) - 7(||a„ - 6^||) - 2!!^ " 



II "r, 



By the uniform weak convexity of Fi with the modulus 7 there exists a point 



w e B. 



liWan — b^ ID + Ctn 



— ) n Fi{t2) C Bs(^\\a„-h\\) 



7(ll«n-6.||) >0. 



CF2{t2 



and 



(3.8) 



\K - w|| < 





a„ + 67r 




a„ + 6^ 




b^- 


2 


+ 


2 


- w 



< -^Wan - &7r|| +7(l|an - b-^W) +"n < IWn - ^tt || • 



Now we put n = n + 1, an = w and again consider cases (1) or (2). 

If the case (2) does not take place for all natural n, then we obtain from the construction of the points 
{an} that the sequence In = ||an — &7r|| satisfies the condition < In+i < ^ + 7('n) + Q!„ < In- It follows 
by the Weierstrass theorem that the sequence In converges to some number I > from the right. Using 
the continuity of the function 7 from the right and taking the limit we deduce that | < 7(Z). The latter is 
possible only in the case I = (see the definition of 7). 

Thus, if for all n S N the case (2) does not take place then H{t2) 9 a„ — > 6^, i.e. 6^ G H{t2)- Taking 
C2 = ^TT we have 

||ci - C2II = ||ci - &7r|| < ||ci - b\\ + \\b- bj,\\ < kuj2 + kuji + k'^uj2. 
The number fc > 1 was arbitrary, hence 

h{H{ti),H{t2)) <uji+2lo2. 

Suppose that for some n e N the case (2) occurs and ||a„ — 67^1! > ||6 — b-j^W. Taking into account that 
||a„ — fe|| > ||a„ — fe^ll — ||6 — 6^11, wc conclude from the inequality of the case (2), that 

(5(||a„ - ^ttII - \\b- 67r||) - 7(l|an - &7r||) < -^ll^ ^ bT,\\. 



From the condition (ii) of the theorem we get 



bA\ <t[ \\\b-b^\ 



< t 



/ kuJl + k^LU2 

\ 2 



By choosing C2 = a„ we obtain 
hiH{h), H{t2)) < ||ci - C2II < ||ci - 6|| + 116 - 6,11 + |la„ - 6, 



,9 f kijJi + k'^u]2 

< kuj2 + kuJi + K"Cl'2 + t 



By taking the limit /c — >■ 1 + 0, we finally prove the theorem. The case |la„ — 6^11 < |16 — ^ttH, which follows 
from the last estimate and from the inequality |j6 — 6,|1 < kuji + fc^W2, also gives formula (SlT)). D 

Remark 3.2. For convex valued mapping F2 a similar result was proved in |3l Theorem 3.1]. 

Remark 3.3. If additionally the conditions of the Theorem 3ll]hold for sets Fiit) and F2{t), then the 
values of the map H in Theorem 3l6]are connected. 

Remark 3.4. In our case the moduli S and 7 are of the second order and we have that t{s) is of the order 
^/s when s — >■ 0. For the Hilbert space this result was proved by Ivanov [10) . 
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4. Application to selection problems 

Theorem 4.1. Let E be a uniformly convex Banach space with modulus Se- Let ^ G E be a collection of 
sets with weakly convex images with modulus of nonconvexity 7(e) (in the sense of Section 3), and suppose 
that all sets from $ are contained in some ball. Let d > 0. Let the condition (ii) holds for moduli dSE(t/d) 
and "f{t). Let dSE(t/d) > j{t) for all admissible t > 0. Suppose that any set H ^ ^ is contained in (each in 
its own) ball of radius r > and 2r < d. 

Then the collection $ has a uniformly continuous selection, i.e. there exists a uniformly continuous in 
the Hausdorff metric function s : ^ ~> E such that for all H £ ^ we have s{H) G H . 

Proof. Without loss of generahty we shall assume that the sets from the family $ are contained in the 
ball Bb.{0) and for any i/ e $ we have g{0,c\ coH) > ri > 0. Consider * = {cl coi/ | if e $}. The 
metric projection of zero y{H) = Pd cohO on the sets from ^ is a uniformly continuous selection defined 
of ^. We have proved in 1?, Lemma 3.1] that for any Hi,H2 S $ (taking into account the condition 
/i(clcoiii,clcoi7i) < h{Hi\Hi)) 

\\y[Hi) - y[H2)\\ < 2h{Hi,H2) + fE{h{Hi,H2)), 

where 

S-'{t/2), t<2AE, 



r s-^{t/2), t< 



Here 5{e) = R6E{e/R), Ag = S{2ri). 

Let y = y{H). Prom g{y,H) < 2r < d using the Theorem 1(1] we conclude that there exists a unique 
metric projection z{H) = PhTJ- 

If 2h{Hi,H2) + fEihiHi,H2)) <id- 2r)/2, then, by defining y, = y{H,), z, = z{H,), z = 1,2, we get 
||2/i-y2||<(d-2r)/2. 

Consider a metric subspace T of the metric space {{E, $), (||-, •||+/i(-, •))). Plements of T are pairs (a;, H) e 
{E, $) such that q{x, H) < d. Consider the set-valued mappings Fi{x, H) = H, F2{x, H) — Bg^x,H){x) from 
T into E. The set-valued mapping Fi is uniformly weakly convex with modulus 7 and uniformly continuous 
with modulus toiit) = t. The set-valued mapping F2 is uniformly convex with modulus dSsis/d). 

Por points (yi, Hi), i = 1,2, we have 

HF2iyuH,),F2iy2,H2)) = hi ~ y2\\ + \g{yi,H,) - ^(2/2,^^2)1 < 
< ||yi -Z/2II + \g{yi,Hi) - g{yi,H2)\ + \g{yi,H2) - g(y2,H2)\, 
\g{yi, Hi) — g{yi, H2)\ < h{Hi, H2), and from the condition ||yi — J/2II < {d — 2r)/2 we obtain that 

Q{yuH2) <\\yi- y2\\ + Q{y2, H2) <{d- 2r)/2 + 2r^{d + 2r)/2 < d. 
Put Z12 e H2: ||?A-2;i2|| = g{yi,H2). Using Corollary 3(1] we get \g{yi,H2) ~ g{y2,H2)\ < \\yi-y2\\ + \\z2- 

2;i2ll < l|yi - 2/2II + tEihi ~ 2/211)- 

Thus in the case 2/i(i/i, 772)-!- /£;(/i(7i^i,i:r2)) < (d—2r)/2 projections z^ = Fi(yi, Hi)r\F2(yi, Hi) uniformly 
continuously depend on sets Hi, i = 1,2, by Theorem 3lSl i.e. 

\\Z1-Z2\\ <Uj{h{Hi,H2)), 

where uj{h{Hi, H2)) is superposition of the function 2h{Hi, H2) + /^(^(-ffi, -^2)) and the function from the 
right side of formula (Si?]). 

If 2h{Hi, H2) + fE{h{Hi, H2)) > {d— 2r)/2, then by the strict monotonicity (increasing) of the function 
fs, there exists a number C > 0, such that h{Hi, H2) > C. In this case 

\\zi-Z2\\<2R<'^h{H^,H2) 

Therefore, s{H) = z(H) is a uniformly continuous selection. D 

Example 4.1. One can apply these results to certain questions about continuous selections of set-valued 
mappings [13], [13 . Let a space E be uniformly convex, a sets A,B G Eh^ such that B is uniformly convex 
with modulus (5(e), e G [0,diami3), and A is weakly convex with modulus 7(e), e G [0,diamyl). Let for 
some d > the inequalities 2diami3 < d and 7(e) < d5E{s/d) for all e G [0,min{2(i, diam A}) hold. Suppose 
that the condition (i) is valid for pairs 5{e), 7(e) and dSE^s/d), 7(e). Then there exist uniformly continuous 
functions a : A + B —i' A and b : A + B —i' B such that for any c G A + B we have a{c) + b{c) = c. 

Proof. By Theorem 3l6]the set-valued mapping A + i? 9 c — >■ H(c) — Br\{c — A) is uniformly continuous. 
By definition the set H(c) is weakly convex with modulus of nonconvexity 7 for all c £ A + B. By the 
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boundedness of the set B, all sets H{c), c G A + B, are contained in some ball. Furthermore for any point 
c G A + B each set H{c) is contained in the ball of radius no larger than 2diami3 < d. By Theorem 4(1] 
there exists a uniformly continuous selection 6(c) = s(H(c)) G B, where the function s(-) is from Theorem 
4(11 a(c) = c - 6(c) € A. D 

5. A CLASS OF WEAKLY CONVEX SETS 

We shall show that simple smooth closed surfaces of codimension 1 are weakly convex sets. In this section 
the space E will be an arbitrary reflexive Banach space. 

We introduce the normal cone N{A, x) to the set A at the point x E A a.s follows 

N{A,x) ^{pe E* I {p,x~- a)> -a^{\\x-a\\) • ||a;-a|| • ||p||, Va G A}, 

where the function ax : [0, diam A) — >• [0, +oo) and limi^+o C(x{t) = 0. 

Let A C E he any closed set with the property cl mtA = A and x € A. Suppose that the set A has the 
following properties: A is path connected, Va; € A 

N{A,x) n B*(0) = {N{A, x) n Bt(0)) |J (iV(cl {E\A), x) n B*(0)) , 

where 

N{A, x) n Bt(0) = M, iV(cl {E\A), x) n Bt(0) = {-p}, 
and there exists infinitely small at zero function a : [0, diani A) — > [0, +oo) with the property 

N{A,x)^{pGE* \ {p,x-a)>-a{\\x-a\\)-\\x-a\\-\\p\\, Mae A], Vx e A, 

N{c\{E\A),x) ^{peE* \ {p,x-a) > -a(||a;-a||)- ||a;-a|| • ||p||, Ma e cl{E\A)}yx e A. 

Then we say that the set A is a smooth closed surface of codimension 1 with a function of smoothness a. 
Roughly speaking, smooth closed surface of codimension 1 is the smooth path connected boundary between 
some set A and its complementary set cl {E\A). 

Let r > 0. Define for any point x G A and for unit vector p G N[A,x) the vector y e E with ||y|| = 
{PtV) = 1- We say that a smooth closed surface A is simple, if for any 2-dimensional affine plane L, such 
that {x,x + 2/} C L, the intersection L n A n Br{x) is a path connected planar curve. We call r > the 
parameter of simplicity. 

Theorem 5.1. Let E he a reflexive Banach space. Suppose that A <Z E is a closed set, cl int A = A, and 
A is a simple smooth closed surface of codimension 1 with the function of smoothness a and the parameter of 
simplicity r > 0. Then the set A is weakly convex with the modulus of nonconvexity 7a (s) < £ (o^l^) + '^(s/2)) 
for all e G [0,min{r, eo})/ where Eq = sup{t > | a(r) + a(r/2) < i, Vr G (0,t)}. 

Proof. Let e G (0,min{r, eo}), xi,a;2 G A such that \\xi — X2II = e. Let pi G N{A,xi) n B^(0) or 
Pi G iV(cl(^\A),a;i)nB*(0). Let Hi = {x e E \ (pi,xi - x) = 0} ^ xi +kerpi. 

Consider Figure 5. Define M = {x E E \ \{pi,xi — x)\ < a(||xi — a;||) • ||xi — a;||}. Using the refiexivity 
of the space E let y G E, \\y\\ = 1, {pi,y) = 1. Let L = afF{a;i,a;i +y,X2}- Let x = ^{xi +2:2), from the 
definition of L and from the definition of smooth closed surface g{x2,Hi n L) = g{x2,Hi) < a{e)e, hence 
g{x,Hi n L) < |a(£). Let xi G i/i n L be a point such that \\x — xi\\ = g{x,Hi n L). 

Let 7 be the connected part of the planar curve L D 3^/2(^1) ^ ^j which intersects the line Hi D L, and 
lies in the same hyperplane with the point x with respect to the line aS{xi,Xi + y}. By the simplicity 
of the surface A and by the inequality e < r there exists z G 7 H A. From the inclusion z E M we have 
g{z,Hi n L) < a(£/2)|. Let zi G i?i n L be a point such that \\z — zi\\ — g{z,Hi n L). 

Choose the right direction of the line HiD L from the point xi to the point w — HiD LO^. 

By the triangle inequality we have from the triangle Xiaiii 

s s s s 

2 - 2"(^) - 11^1 - '^ill < 2 + 2^^^^ 

and from the triangle xizzi 

2-r(2)^"l^^-^^ll^2 + 2n2)- 
If the point xi lies to the right of the point w then 

\\xi - w\\ ^ \\xi - xi\\- \\xi - w\\ < -a(e). 



2 



If the point xi lies to the left of the point w then 



\xi - w\\ = \\xi - w\\ - \\xi - xi\\ < -a{e). 
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xi + y 




Figure 5: Proof of Theorem 5.1 



In both cases ||ii — w|| < ^a{e). 

In the same way we obtam that ||zi — w\\ < |a (|). 
From this we deduce that 

\\zi ~ii\\< \\ii ~ w\\ + ||zi - w\\ < -a{e) + -a \^-j . 
Finally, for any a;i,X2 G A, \\xi — X2\\ — e, x ^ 5(^1 '^ ■^2) there exists z £ A such that 
\\i - z\\ < \\x -ii\\ + \\z -zi\\ + \\xi - zill < £ (a(£) + a (I)) < f • 

D 
We observe that under the conditions of Theorem 5ll]both sets A and cl {E\A) are weakly convex. The 

proof easily follows from Theorem 5lT] 

Let E he a. Banach space and a subset A C E he closed. We shall say that unit vector n G E is a proximall 

normal to the set A at the point x G A if there exists r > such that 

AnmtBr{x + rn) = 0. 

Theorem 5.2. Let space E be uniformly convex with modulus of convexity of the second order and 
uniformly smooth, and let subsets A d E and cl {E\A) be weakly convex with modulus 7(e) of the second 
order and cl int ^ = A. Then N{A,x) n 5*(0) = {pix)}, N{cl{E\A),x) n 5*(0) = {-p{x)}, at any point 
x € A and p{x) uniformly continuously depends on the point x G A. 

Proof. By Remark 2l2]the sets A and cl {E\A) are proximally smooth with some parameter d> {). Using 
the results of Ivanov |1H Theorem 2] we have that for any point a; € A there exists a proximally normal 
vector n{x) to the set A at the point x S A (and proximally normal vector —n{x) to the set cl {E\A) at the 
point x e A) and n{x) uniformly continuously depends on x. By uniform smoothness of the space E for any 
vector n (£ E, \\n\\ = 1, there exists unit vector p(n) £ E* with (p(n),n) = 1 andp(n) uniformly continuously 
depends on n ([9l[T4]). Again by the smoothness of the space E we have that p{x) = p{n{x)) G N{A,x) is 
uniformly continuous on a; e A. D 
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6. Epilogue 



1. We see from the results above that in the spaces with moduhis of convexity of the second order the 
notion of weakly convex set is very effective. 

2. Some of the results can be proved in a more general setting of uniformly convex Banach spaces: 
Theorem 3|31 or Theorems 3[T1 SUl and 3I3] (see [5] for details). However, the proofs in f? are much more 
complicated. 

We wish to point out that the results above are interesting and nontrivial even in the finite-dimensional 
case. 

3. We agree with Banas that the modulus a from [4j and [Sj is sometimes much more convenient in 
applications than the standard modulus of smoothness [5], [T3]. In fact, the modulus of nonconvexity is a 
modification of modulus a from [i] for the nonconvex case. 

Also, Theorem 5lT]shows a deep relationship between weakly convex sets and smooth sets. In conclusion, 
we formulate the following 

Conjecture 6.1. Let the space E he uniformly convex (and smooth?), the subsets A <Z E and c\{E\A) 
closed and weakly convex with modulus 7(e) and cl mi A = A. If lim ^{e)/e = 0, then the unit normal 

vector to the set A at the point x ^ A uniformly continuously depends on the point x G A. 
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